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To A. A. Kirillov on his 60th birthday 

, Abstract. Let G be any of the complex classical groups GL{n) , SO{2n + 1) , 

Sp{2n) , 0{2n) , let g denote the Lie algebra of G, and let Z{q) denote the subal- 
gebra of G -invariants in the universal enveloping algebra U{g) . We derive a Taylor- 
^ ' type expansion for finite-dimensional characters of G (the binomial formula) and use 

, it to specify a distinguished linear basis in Z{q) . The eigenvalues of the basis ele- 

ments in highest weight g-modules are certain shifted (or factorial) analogs of Schur 
functions. We also study an associated homogeneous basis in /(g) , the subalgebra of 
G -invariants in the symmetric algebra S{q) . Finally, we show that the both bases 
, are related by a G -equi variant linear isomorphism a: /(g) — > Z{q) , called the special 

0^ ' symmetrization. 

> 

^ ■ The present paper is a continuation of our previous work [001] but can be read 

independently. Here we aim to transfer to the orthogonal and symplectic groups a 
part of results of [001] connected with the general linear group. 

Let G be one of the groups SO{2n -fl, C) , Sp{2n, C) / let q be the Lie algebra 
of G, U{q) be the universal enveloping algebra of g, and Z{g) be the center of 
U{g) . The commutative algebra Z{g) is also known as the algebra of biinvariant 
differential operators on the group G , or the algebra of Laplace operators. 

The irreducible finite-dimensional representations of the group G are indexed 
by partitions A of length ^ n ; we denote them by Vx and we view each Vx also 
as a t/ (g) -module. 

Our main object is a distinguished linear basis {T^} C Z{g). Here the index 
fj. ranges over partitions of length ^ n and each basis element can be char- 
acterized, within a scalar multiple, as the unique element in Z{q) of degree 2|/i| 
satisfying the following vanishing condition: the eigenvalue of in V\ equals 
if |A| ^ \n\ and X ^ n (moreover, it turns out that the eigenvalue is if Ai < fXi 
for at least one i). 



Introduction 
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^In the present Introduction, to simplify the discussion, we exclude the even orthogonal group 
SO{2n, C) . However, after minor modifications, all our constructions hold for this group as well. 
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For general A, we denote the eigenvalue of in Vx by t* (A) = t* (Ai, . . . , A„) . 
The functions t* are inhomogeneous polynomials with a certain kind of symmetry; 
they can be identified with certain factorial analogs of the Schur polynomials. 
Note that the top homogeneous component of the inhomogeneous polynomial 

coincides with a Schur polynomial in the squares of the arguments: 

tfj, {xi , . . . , Xfi) = Syu {xi , . . . , x^) . 

The polynomials t* arise in the binomial formula for the characters of G . De- 
note by x\ = Xa(-21) • • • ) -^n) the character of V\ ; here zi, . . . , Zn are natural co- 
ordinates of a maximal torus of G. Assume 'additive' variables 
connected with the 'multiplicative' variables zi, . . . ,Zn by the relation 

Xi = z^/^ - z~^/^, i.e., x'^ = z + z~^-2. 

The binomial formula is written as 

Xxjzi, ■ ■ ■ , Zn) _ t*(Ai,...,An)t^(xi,...,a;n) 
Xa(1, •••,!) V c±(n,//) 

where c± (n, //) are simple normalization factors. This is a Taylor-type expansion 
for the character near the unit element of the group. ^ 

One more important property of the basis elements is existence of a relation 
between the bases corresponding to different values of the parameter n. We call 
this the coherence property of the canonical basis. 

Then we study a counterpart of {T^} for the algebra /(g) , the subalgebra of 
G-invariants in the symmetric algebra S{q) . Note that I{q) is a graded algebra 
which is canonically isomorphic to grZ{Q), the graded algebra associated to the 
natural filtration in Z{q) . 

To the basis {T^} corresponds a homogeneous basis {T^} of each 
coincides with the leading term of (with respect to the identification = 
gr Z{g) ). On the other hand, the basis {T^} can be described without reference to 
Z{g) , in intrinsic terms of the G-module S{q) . Note that the elements admit 
an explicit expression as polynomials in natural generators of the Lie algebra Q . 

Finally, we show that there exists a G-equi variant linear isomorphism 

a:Sig)^Uig) 

which preserves leading terms and takes to for any /x . We call a the special 
symmetrization for the orthogonal/symplectic Lie algebras. 

The special symmetrization a enters a family of 'generalized symmetrizations' 
studied in the paper [02]. The results of [02] provide explicit combinatorial for- 
mulas both for a and its inverse . Since we dispose with an explicit expression 
for , this yields a certain formula for the elements = • 

We conclude this Introduction with a brief discussion of some related works. 

For the general linear group GL{n, C) , the binomial theorem and distinguished 
bases {§^} C Z{Ql{n, C)) and {S^} C I{gi{n, C)) with similar properties were ear- 
lier considered in [001]. The elements S^, which are called quantum immanants, 



■^Note that the binomial formula for characters of classical groups of type B, C, D is a 
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appear in a higher version of the classical Capelli identity and admit a remarkable 
explicit expression, see the papers [Okl, N, Ok2]. The eigenvalues of the elements 
§^ in highest weight modules are described by certain polynomials s* (Ai, . . . , A^) , 
called in [001] the shifted Schur polynomials. The both families of polynomials, 
{s* } and {t* } , have similar properties.^ 

Let fi be of the form (1™) for the orthogonal group or of the form (m) for the 
symplectic group (m = 1,2,...). Then the corresponding elements coincide 
with Laplace operators considered by A. Molev and M. Nazarov in [MN]. As shown 
in [MN], these elements also occur in a Capelli-type identity and admit an explicit 
expression in terms of the generators of the Lie algebra. An open problem is to 
generalize the results of [MN] to arbitrary Note that the explicit formulas 
found in [MN, Okl, N, Ok2] differ from the formulas obtained via the special 
symmetrization. 

Many of the results of [001] and the present paper should have counterparts 
for classical Lie superalgebras Qi{p\q) , and osp(?i|2m). Some work in this 

direction was made by A. Borodin and N. Rozhkovskaya [BR], A. Molev [Mo], 
V. Ivanov and A. Okounkov (see [I]). 

In the present work (as in [001]), we used very elementary tools such as explicit 
formulas for the (ordinary and factorial) Schur polynomials or Weyl's character 
formula. We believe our approach can be further developed by making use of 
a more fine technique. In particular, there exist more involved versions of the 
binomial formula, see [002], [Ok3], and [Ok4]. 

§0. Main notation 

Unless otherwise stated, the symbol G{n) or G will denote any of the complex 
classical groups of rank n 

GL{n,C), Sp{2n,C), SO{2n+l,C), S0{2n,C), 

which constitute the series A, C, B, D, respectively. By g{n) or g we denote the 
corresponding complex Lie algebras. In the case of the series D we also consider 
the nonconnected groups G' = G'{n) = 0{2n, C) . 

By U{q) we denote the universal enveloping algebra of g. For the series A, C, 
B, we denote by Z[q) the center of U{q) (which coincides with the subalgebra of 
G -invariants in U{q) ); for the series D the same symbol will denote the subalgebra 
of C -invariants in U{q) (which is a proper subalgebra of the center). 

Similarly, we define /(g) C S'(g) as the subalgebra of G-invariants (or G' - 
invariants, for the series D) in the symmetric algebra of g. 

V\ or V\\n is the irreducible finite-dimensional complex- analytic representation 
of G with highest weight A; it is also viewed as a [/(g) -module. 

C [G] is the space of regular fvmctions on G , where G is viewed as an algebraic 
group over C ; in other words, C [G] is the linear span of matrix elements of all 
representations V\ . 

By fi we always denote a partition of length l{fj,) ^ n; it is also viewed as a 
Young diagram; |//| denotes the number of boxes of the diagram /i. 
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§1. Binomial formula 
Here we derive a multidimensional analog of the binomial formula 

k 

^^—k{k-l)---{k-m + l)a;'", 



where the powers of a variable will be replaced by characters of G . 

Let us agree about the choice of a Borel subgroup B G G and a maximal torus 
H <Z B (thus positive roots and dominant weights will be specified): 

• For the series A, we take as B the subgroup of upper triangular matrices and 
as H the subgroup of diagonal matrices, 



H = {diag{zi,...,Zn)}, 



• For the series C-B-D we identity G with a subgroup in GL{N,C), where 
N = 2n OT N = 2n+l, 



G^{geGL{N,C)\g'Mg^M}, 



(1.1) 



where M stands for the following symmetric (case B-D) or antisymmetric (case 
C) matrix of order N : 



M 



1 



±1 



±1 



(1.2) 



Then as B and H we take the subgroups of upper triangular or diagonal matrices 
in G. Thus 

H = {diag{zi, . . . , Zn, z~'^ , . . . , z^^)}, case C-D 

H = {diag {zi,...,Zn,l,z~^,---, Zi^)}, case B . 

For all the series, the weights of H are of the form 

z^ = z^^ ... where A = (Ai, . . . , An) e Z^, 

while the dominant weights are distinguished by the supplementary conditions 

Ai ^ • • • ^ An, case A, 

Ai ^ • • • ^ An ^ 0, case C-B, 

Ai ^ • • • ^ An-i ^ |An|, case D . 



The dominant weights for the series A are called the signatures and those for 
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The irreducible character of G, indexed by a dominant weight A (i.e., the char- 
acter of V\ = V\\n) will be denoted by the symbol 

ql(n) sp(2n) so(2n+l) so(2n) 

Xx=Xx , Xx ' Xa or Xa • 

For the series D, it is more convenient to deal with the reducible character 

o{2n) _ so(2n) so(2n) wliPrP \. > . . . > \ > fl 

Xa - Xai,...,a„ + Xai,...,a„_i -A„' wnere Ai ^ • • • ^ An ^ u . 

When Xn — 0, this is simply 2%^°''^"^'' , and when > , this coincides with the 
restriction to the group G — SO{2n,C) of an irreducible character of the group 

G' = 0{2n,C)- 

Thus, we are working with the characters 

gl{n) sp{2n) so(2n+l) o(2n) 

Xa ' Xa ' Xa ' Xa ' 

where A is either a signature (case A) or a positive signature (case C-B-D). 
To each A we assign another weight Z = (Zi, . . . , Z^) defined as 

I = (Ai + n — 1, A2 + n — 2, . . . , An), case A, (1-3) 

Z = (Ai + n - 1 + £, A2 + n - 2 + £, . . . , An + £), case C-B-D, (1.4) 



where 



1, case C 



1/2, case B, (1.5) 
0, case D . 

For the series C-B-D, Z just equals A plus the half-sum of positive roots, and for 
the series A, the same holds after restriction to SL{n, C) C GL{n, C) . 

Unless otherwise stated, we shall consider characters of G as functions on the 
torus H, i.e., as functions of the coordinates zi,...,Zn- The following formu- 
las follow from general Weyl's character formula (the determinants below are of 
order n): 

xf"'(.....,.„)-^'?^ 



so(2n+l)/ N 
Xa {Zl,---,Zn) 



'^i<j{Zi — Zj) 



l^ ' 



Tli<j{Zi + Z^ -Zj-z^ ) 

soi2n^i)u^ . ^ - det[(4--.-'-)/(.j/^-.7V^)] 



^^<j{Zi^Z^ ^ -Zj-Zj ^) 

Note that Xx^^\^p^ • • • ' ^n) — sx{zi, . . . , z^) , the Schur polynomial in n vari- 
able, if A is a partition. The characters x^x^"^^ symmetric Laurent polynomials 
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variables Zi + , i = 1, ...,n. Note that the last claim fails when X\'^^^ is 
replaced by x^"*^^"^"* ; this is the reason why we deal with the characters X\^^^ ■ 

Let a = (ai, a2, . . . ) be an arbitrary number sequence. The generalized factorial 
powers of a variable x are defined as 



{x\ay 



{x-ai)---{x- afe), k^l, 
1, k = 0. 



When a = these are the ordinary powers, when a = (0,1, 2,...) these are the 
falling factorial powers, and when a = (0,-1, —2, . . . ) these are the raising factorial 
powers (or the Pohgammer symbol). 

Further, the generalized factorial Schur polynomial in n variables, indexed by a 
partition with /(/x) ^ n, is defined as 

det[(xj-|a)^^+'^-*] _ det[(xj|a)'^^+"-^] 

\Z\ 5 ... 5 Xyi I a,) 



det[(a;j|a)'* Y{i<Axi-Xj) 



When a = this turns into the standard formula for the ordinary Schur polyno- 
mial s^{xi, . . . , x„) , and in the general case s^{xi, . . . , | a) is an inhomogeneous 
symmetric polynomial of degree = /^i + • • • + /in and with the top homogeneous 
component equal to s^(xi, . . . , Xn) : 

, Xfi I o) — •S/i(a;i, . . . , Xn) + lower terms . 

It follows that the polynomials s^(a;i, . . . , a^n | o) form a basis in the algebra of 
symmetric polynomials in n variables 

The polynomials s^(a;i, . . . , 1 0, 1, . . . ) were introduced by Biedenharn and 
Louck and called the factorial Schur polynomials, see [BLl, BL2]. The general 
definition is due to Macdonald, see [Ml] and [M2, 1, §3, ex. 20] (note that Macdonald 
uses the notation [x \ a)'^ = {x + ai) ■ • • {x + ak) )■ 

Theorem 1.1 (Binomial formula for GL{n)). Assume 

Zi = l+Xi, Zn=l + Xn. 

Then 

Xt^''\zi, ...,Zn) S^{h, . . . , I 0, 1, 2, . . .)s^{xi, ...,Xn) 

xf^)(i,...,i) ir ^(^'^) 

Here A = (Ai, . . . , A„) is an arbitrary signature, li = Xi + n — i (i = 1, . . . , n), jj, 
ranges over partitions of length ^ n , and 

.(»,/.)= n ("+i-')=n '''fa":;" . d-n 

where the first product is taken over all boxes {i, j) of the Young diagram repre- 
senting the partition n . 

Comment. This version of the binomial formula was proposed by the authors, see 
[001, Theorem 5.1]. It is equivalent to the expansion 

sa(1 + a^i, . . . , 1 + a;„) = ^ dx^s^{xi, ...,Xn), 
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where the coefficients dxf^ are given by 

dxpi = det 



Xi + n — i 



see Lascoux [Lasc], Macdonald [M2, I, §3, ex. 10]. 

Although the latter formula looks simpler than (1.6), formula (1.6) has a number 
of advantages, as it is explained in our paper [001]. E.g., an important idea is to 
consider Zi,...,/^ as variables rather than parameters, and then formula (1.6) 
makes in evidence the fact that the roles of li, . . . ,1^ and of xi, . . . ,Xn are almost 
symmetric. 

Theorem 1.2 (Binomial formula for the series C-B-D). Assume the variables Zi 
and Xi are subject to the relations 

2 _ I -1 _ 9 • ur ^/^ _ • _ 1 

Xj^ — Zi ~t~ Zi Z, Z. C, Xi — zizyZ^ Zj^ j , % — J. , . . . , 77- . 

Let G be any of the groups of the series C, B, D and let us abbreviate x\ for 
^sp(2n) ^ ^so(2n+i) X\^^^ i whcrc A is a partition of length ^ n. Then we have 

Xx{zi, . . . , Zn) _ S^(/f,...,/2 |£2^(£+ l)2,...)s^^^a^|^^^^^^ 



Xa(1, •••,!) V c±{n',fJ') 

Here = Aj + n — z + e, e is defined by (1.5), ji ranges over partitions of length 
^ n , and 

c±(n,//)= J] 4(n + j-i)(n±l/2 + j-i), (1.9) 

(*,j)eM 

where the sign " + " is taken for the series C, B, while the sign " — " is taken for 
the series D. 

Formula (1.8) is a particular case of the expansion of a type BCn Jacobi poly- 
nomial with "Jack parameter" a = 1 into a series of Schur polynomials. This ex- 
pansion probably is well-known to experts (sec, e.g., Lassalle [Lass, Theoreme 9]). 
For sake of completeness we present a detailed proof. 

Proof. Consider the classical Jacobi polynomials p^'^^ and put 

p{oc,(3) _ p{oc,l3) /p(«'/3)n~\ 

Let Tfe and Uk he the Chebyshev polynomials of the first and second kind, respec- 
tively. Then for any A; = 0, 1, . . . 

- Z-'^-y^ _ ^(1/2,-1/2)^^ + ^-^ 

-i J 



^1/2 _ ^-1/2 k \ 2 
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where the symbol " ~ " means equality within a number factor depending on k but 
not depending on z (see Szego [S, (4.1.7) and (4.1.8)]). 

Substituting these expressions into the formulas for the characters XA given 
above we obtain 

, , det[Pir45_,(l + t,)]i^,,^„ 

where 

_i r (1/2,1/2), caseC, 

ti = - 1, i = l,...,n, («,/?)- Ul/2, -1/2), caseB, 

[ (-1/2,-1/2), case D, 

and the symbol "~" means equality within a factor not depending on ti, . . . ,tn- 
Recall a well-known identity: for any power series 

oo 
m=0 

we have 

'^'^-'f'^'f^" = Y: det [a«,„_,] (1.11) 

where /j, ranges over partitions of length ^ n (see, e.g., Hua [Hua, Theorem 1.2.1]). 
We shall apply this identity to 

m = Pi:fhi^ + t), i = l,...,n, 
and we shall use the well-known expansion 

p{a,/3)r-. , V Hk - 'i-) ■ ■ ■ jk - m + l)(k + a + P + 1) ■ ■ ■ (k + a + P + m) 
^ ^ ' ^ 2^m! (a + 1) •••(« + m) 

(see, e.g., Szego [S, (4.21.2)]). 

Remark that in our situation a -|- /3 -|- 1 = 2£ , so that 

(/c - z)(/c + a + /? + 1 + z) = (/c + e)^ - (£ + if, z = 0, 1, . . . . 

Further, remark that a = ±1/2 according to the assumption on the sign made in 
the statement of the theorem. It follows that the above expansion can be rewritten 
as 

p(a,/3) n + .^ = V iik + ef\eMe + l)^...rt^ 
k ^ ^ 2^m\ (±1/2 + 1) ■■■(±1/2 + m)' ^' ' 

Formulas (1.10), (1.11), and (1.12) imply that 

Xx{zi,...,Zn) 

det [(/f I e^, {e + 1)2, . . . )M.+n-j] . . . , t^) 



E 



n'" , (2i^i+^-i(ij.. -4-n-i)\(+^/2 + ^)■■■ (+^ 12 lu n - iX\ ' 
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The normalizing factor in this formula is determined by the condition that the 
constant term (i.e., the coefficient of S0{ti, . . . ,tn) = 1) should be equal to 1. 
After simple transformations, using the relations 

• • • 7 ^n) — 2 ^^^Spii^Xi, . . . , 2;^), 

det[(^f|£^(5+l)^■■■r^+-^•] _ , ,21,2. ,.^2 ^ 

det[(/f|£2,(£ + l)2,...)-^] -«Ml'l---'nl^ + ^) ^■■■)^ 

we obtain the desired formula. □ 

§2. A DISTINGUISHED BASIS IN Z{q) 

The aim of this section is to construct and characterize a linear basis in Z{q) . 
Our construction is suggested by the binomial formula of §1. 

Let Oe{G) be the algebra of germs of holomorphic functions at the unity e of 
the group G and let M-dG) be the maximal ideal of Oe{G) formed by germs 
vanishing at e. We may identity U{q) with the space of those linear functionals 
on Oe{G) that vanish on Aie{G)"^ , where m is large enough. This defines a 
nondegenerate pairing ( • , • ) between U{q) and C [G] (see §0 for the definition of 
C[G] ). If F is a finite-dimensional G -module (also viewed as a t/ (0) -module) , V* 
is the dual module, $, & V and rj e V* are arbitrary vectors, and f^r){g) = vidO 
is the corresponding matrix coefficient, then we have 

{X,f^r,) = v{XO for any X e t/(0) . 

Let I{G) C C [G] be the subalgebra of invariants of the group G (for series A, C, 
B) or the group G' (for series D) with respect to its action on G by conjugations; 
note that this definition is parallel to that of Z{g) C U{g) , see §0. 

Remark that both U{g) and C [G] are semisimple modules over G or G' . It 
follows that there are canonical projections 

#: U{g)^Z{Q), (2.1) 
#:C[G']^/(G'); (2.2) 

these projections also can be defined as averaging over a compact form of the group 
G or G' . The pairing ( • , • ) over U (g) and C [G] is invariant over G or G' and 
so defines a nondegenerate pairing between Z{q) and I{G) . 

Lemma-Definition 2.1. Let jj, range over the set of partitions of length ^ n. 
There exist central elements 

§^ G ^■(g), degS^^|;u| {series A), 

e Z(0), degT^ ^ 2|//| {series C, B, D), 



uniquely specified by the condition 

{^,,,f)s^{xi,...,Xn) 
c(n, //) 



{f* |^)(zi,...,Zn) = ^ 

if* \H){zi,---,Zn) = V . . 

' c-i-ln. 11.) 
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where H <Z G is the diagonal torus defined in §1, / G C[G] stands for an arbi- 
trary test function^ e I{G) is the image of f under the projection (2.2), and 
the variables xi,...,Xn and the normalizing factors c{n, fi) , c±{n, fi) are given 
by{1.6), (1.9). 

Proof. The function f^ |^ is a symmetric Laurent polynomial in zi,...,Zn', in 
the case of the series C, B, D it is also invariant under transformations Zi ^-^ . 
It follows that can be expanded into a series of polynomials s^(xi, . . . , Xn) 

or s^(xf , . . . , x^) , where Xi = Zi — 1 (for scries A) or = Zi + z^^ — 2 (for series 
C, B, D). Given the map assigning to / the nth coefficient in that expansion 
is a linear functional on C [G] which only depends on a finite jet of / at the point 
e G G (the order of the jet is equal to \fx\ or 2|(u|). Hence this functional is an 
element of the algebra U{g) of degree ^ |//| or ^ 2||u|. This proves existence of 
the elements S^u or . As will be clear in what follows, we have exact equalities 

deg§^ = |//|, degT^ = 2|//|. 

Finally, the elements , belong to Z{q) , because their values at any test 
function f E C[G] depend on only. □ 

Recall that Vx denotes the irreducible G -module with highest weight A = 
(Ai,...,A„), which is equally viewed as a L^(0) -module. Any central element 
X e U{g) acts in V\ as a scalar operator const -1; the corresponding constant 
is called the eigenvalue of the central element X in Vx. 

Theorem 2.2. Let G = GL{n,C) . The elements defined in Lemma 2.1 form 
a basis in Z{g) and are characterized (within a scalar multiple) by the following 

properties: 

(i) degS^ ^ ; 

(ii) the eigenvalue of in a module Vx equals for all partitions A such that 
1{X) ^ n, \X\ ^ Inl , A ^ 

(iii) the eigenvalue of in V^ is nonzero. 

Theorem 2.3. Let G be a classical group of type C, B, D . The elements 
defined in Theorem 2.1 form a basis in Z{g) and are characterized (within a scalar 
multiple) by the following properties: 

(i) degJ^ ^ 2|yLt| ; 

(ii) the eigenvalue of in a module Vx equals for all partitions A such that 

1{X) ^ n , \X\ ^ \ , X fx; 

(iii) the eigenvalue of in V^ is nonzero. 

Comment. Note a difference between these two claims: in Theorem 2.3 we used 
all irreducible G-modules"^ to characterize our basis, while in Theorem 2.2 we used 
only a part of the modules (namely, the polynomial ones). This implies that for the 
series A, our basis {S^} is not invariant under the outer automorphism of 0l(n, C) . 

To prove Theorems 2.2-2.3 we shall restate them as purely combinatorial claims. 
We shall need the Harish-Chandra homomorphism. Recall (see, e.g., Dixmier [D, 

^The fact that for the series D there exist dominant highest weights A = (Ai, . . . , A^) with 
An < is not relevant here, because for any element of Z(o(2n)) , its eigenvalue does not change 

_1 _ J C \ \ 
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7.4]) that it is defined for any reductive Lie algebra Q over C. Fix a triangular 
decomposition g = n_ ® 1^ © n+ where 1^ is a Cartan subalgebra and n+ and n_ 
are spanned by positive and negative root vectors. The Harish-Chandra homomor- 
phism maps [/(g)^ , the centralizer of i) in U{g) , onto U{[)) = S{i)) . Let us identity 
with C[f)*] , the algebra of polynomial functions on the dual space ()* . It is 
known that the restriction of the Harish-Chandra homomorphism to the center of 
U{q) is injective and its image in C[l^*] consists of those polynomials /(A) on f)* 
that are invariant under transformations 

X^w{X + p)-p, Aef)*, (2.1) 

where w ranges over the Weyl group W — W{Q,i)) and p denotes the half-sum 
of positive roots. Moreover, if X is a central element of U{q) and fx is the 
corresponding invariant polynomial then the value of fx at a point Aef)* coincides 
with the eigenvalue of X in the irreducible highest weight module over g with 
highest weight A. Although this is true for all A G [)* we shall deal with dominant 
weights A only; note that any polynomial /(A) is uniquely determined by its values 
on the set of dominant weights. Finally, note that deg X = deg fx ■ 

After this digression let us return to the classical Lie algebras. Examine first 
the series A. Then the group W coincides with the symmetric group S{n) and the 
weight p satisfies the property pi — pi^i = 1, l^i^n — 1. It follows that a 
polynomial /(Ai, . . . , A„) is invariant under the action (2.1) if and only if it satisfies 
the following symmetry property: 

/(Ai, . . . , An) = /(Ai, . . . , Ai_i, Ai+i — 1, Aj -I- 1, Ai-i-2, • • • , An) (2.2) 

for i = 1, . . . , 71 — 1 . In our paper [001] such polynomials were called shifted 
symmetric ones and the algebra of shifted symmetric polynomials in n variables 
was denoted by A*(n) . Clearly, /(Ai, . . . , An) is shifted symmetric if and only if 
it is symmetric in the variables li, . . . ,1^ that are related to Ai, . . . , An by (1.3). 
Note that the top homogeneous component of a shifted symmetric polynomial is 
a symmetric polynomial. The Harish-Chandra homomorphism induces an algebra 
isomorphism Z{gl{n)) A*(n) . 

Next assume G is one of the classical groups of type C, B, D. Then the weight 
p has the form 

p = = (n — 1 + £, n — 2 + £, . . . , £), where £ = 1, 1/2, 0, 

and the group W is either the hyperoctahedral group H{n) = S{n) k {±1}" (C-B 
case) or its subgroup H'in) C H{n) of index 2 (D case); recall that the action 
of the generators of the subgroup {±1}"^ is of the form Aj h- > — Aj . However, for 
the series D, we have replaced the center of U{o{2n, C)) by its proper subalgebra 
Z{o{2n, C)) , which is just equivalent to replacement of H'{n) by the whole group 
H{n) . This allows us to describe the algebra Z{q) in a uniform way for the three 
series C, B, D. 

Denote by M*(n) = M*(n) the algebra of polynomials in Ai, . . . , An that can 
be written in the form 

/(Ai, . . . , An) = g{ll, 1 = X + Pe, 

where g is an arbitrary symmetric polynomial. Then we obtain an algebra isomor- 
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Lemma-Definition 2.4. For the series A, put 

S* (Ai, . . . , An) = S/i(il, . . . , Zn 1 0, 1, 2, . . . ) 

and for the series C, B, D, put 

^;(Al,...,An) = s^(^?,...,/^|£^(£ + l)^...), 

where e = 1, 1/2, 0, respectively. The Harish-Chandra homomorphism sends §^ to 
s* and to . 

Proof. This foUows at once from the above discussion if one compares the definition 
of the elements , (Lemma-Definition 2.1), the claim of Theorems 1.1-1.2, 
and the above definition of s* and t* . □ 

Since the map Z{gl{n)) A*(n) preserves degree and since degs* = , we 
have degS^ = Ia*I • Similarly, since the top homogeneous component of t^(Ai, . . . , An) 
coincides with S|_^{Xl, . . . , A^) , we have deg t^ = 2 , so that deg = 2 \n\ . 

Now it is clear that Theorems 2.2-2.3 are equivalent to the following claims 
about the polynomials s* and t* . 

Theorem 2.5. The polynomials (Ai, . . . , An) and t* (Ai, . . . , An) form a basis 
of the algebras A*(n) and M*{n), respectively. They can be characterized (within 
a scalar multiple) by the following properties: 

(i) s* G A*{n) and dcgs* = |/i| , t* G M*(?i) and degt* =2\n\; 

(ii) s*(A) = and t^(A) = for all partitions A such that 1{X) ^ n, |A| ^ |//| , 

(iii) s*^{fi) ^ and t*^{^) ^0. 

Proof. The part of these claims concerning the polynomials s* was established in 
[Okl]; see also [001, Theorem 3.3]. So we shall only consider the case of t* (which, 
however, is quite similar to that of s* ). 

Let M(n) denote the algebra of polynomials in n variables Ai, . . . , An , invariant 
under the natural action of the hyperoctahedral group H{n) (the generators of 
{±1}"^ C H{n) change the sign of the variables). Clearly, the polynomials 

tn{Xi, . . An) := 'S/i(Ai, . . . , A^) 

form a homogeneous basis in M(n) . 

On the other hand, M(?i) is canonically isomorphic to the graded algebra asso- 
ciated with the filtered algebra M*(?i) . Since coincides with the top component 
of , we conclude that the polynomials t* form a basis in M* (n) . 

Let us check that the polynomials i* satisfy the conditions (i)-(iii). The con- 
dition (i) is evident and the conditions (ii), (iii) will be deduced from the explicit 
formula 

^;(Al,...,An) = sp(/^...,/^k^(£ + l)^...) 

_ det[(/f |£^(£+l)^...)^^+'^-^^] 
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Note that the denominator is nonzero, because , . . . , strictly decrease. Thus, 
we have to analyze the vanishing properties of the numerator. 

Assume |A| ^ \iJ,\ and \ ^ ji. Then there exists k e such that 

< l^ki whence 

Xi < jjLj for j ^ k . 
For such couples i, j the {i,j)th entry of the determinant in the numerator is 

(Z2|£^(e+l)2,...)w+"-^= H ((X, + n-i + ef-{e + r-lf) = 0, 

r=l 

SO that the determinant vanishes. 

If A = /U then a similar argument shows that the matrix in the numerator is 
strictly upper triangular with nonzero diagonal entries, so that the determinant is 
nonzero. 

Thus, we have verified the properties (i)-(iii) and it remains to prove that poly- 
nomials with such properties are unique. For d = 2, 4, 6, . . . consider the subspace 
M^(n) C M*(n) formed by elements of degree ^ d. The existence property verified 
above implies that the linear functionals on M^(n) of the form 

/ ^ /(A), |A| ^ d, 

are linearly independent. On the other hand, their number just equals dimM^(n) , 
which implies the uniqueness claim. □ 

Note that Theorem 2.5 also can be directly deduced from the binomial formula, 
cf. second proof of Theorem 5.1 in [001]. 

§3. Coherence property 

Until now we fixed a classical Lie algebra g . In this section we deal with the whole 
series {^(n)} of classical Lie algebras (of type A,C,B or D). We define natural 
embeddings Z{Q{n)) Z{g{n + 1)) and we show that the elements of the canonical 
bases are stable (within scalar multiples) with respect to these embeddings. 

Since the Lie algebra will vary (inside a classical scries), we shall use more 
detailed notation for the polynomials in n variables introduced above: 

* J. J.* 

^H\ni ^ix\ni Vl'^' ''/i|n' 

and the elements of the distinguished basis of Z{g{n)) will be denoted by S^|^ or 

For any n, we shall identify C" with the subspace of C""''-'^ spanned by first 
n basis vectors. Since Q{n) is realized in (for series A) or in C"^ © C"^ (for 
series C, B, D), we obtain a natural Lie algebra embedding Q{n) 0(^1 + 1) , which 
induces a natural algebra embedding ?7(g(n)) — >■ U (^(n + 1)) . 

Next, we define the averaging operator 

Avr„,„+i : Z{Q{n)) Z{Q{n + 1)) 

as the composition 
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where the latter arrow is the projection ^ defined in (2.1). Similarly, for any couple 
n < m, we define the averaging operator 

AvTnm- Z{Q{n)) Z(0(m)), 

which also coincides with composition of averagings 

Zigin)) ^ ZiQin + 1)) ^ • • • ^ Z{g{m)) . 

Theorem 3.1. Let c{n, fi) and c±{n, n) be the normalizing factors occurring in 
the binomial formula {see (1.7), (1-9)). For any n < m we have 



Avr^ 



1 



(c±(n,//) ^'^l") " ^^{^) ^^^"^ • 

We call this the coherence property of the basis {§^} or {T^} . For the series A 
the coherence property was established in our paper [001, §10], by three different 
methods which can also be transferred to the series C, B, D. We shall give now one 
of the proofs, which is based on the binomial formula. 

Proof. We assume {^(n)} is one the series C, B, D. First of all we shall restate 
Theorem 1.2. Let g E G and X e Q he related by 
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l/2_^-l/2=X, 



where g is close to e e G and X is close to e g . Let f E I{G) be a test function 
and let /' = f'{X) be the corresponding function in a neighborhood of e g. 
Write ±a;i, ±a;2, • • • for the eigenvalues of the matrix X and define 

T,j,{X) = s^{xl,xl,...), X eg, l{ij) ^ rankg. 

One can expand f\X) into a series of the invariant polynomials Tp(X), which 
converges in a neighborhood of the origin. We claim that this expansion can be 
written as 

f'{x)= Yl (3.1) 

i(/x)^rank Q 

where 

•= f \ Y^M- 

c±(rankg, (U) 

Indeed, since I{G) is spanned by the characters x\ (here we use the same 
abbreviation as in Theorem 1.2), we may assume / = Xx/Xx{^)- But then our 
claim is just a restatement of the binomial formula. To see this we have to recall 
the definition of the elements and use the fact that for any element A G Z{q) 
(in particular, for A = T^), (A, x/x(e)) equals the eigenvalue of A in the U{q)- 
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Now let n < m and let G{n) ^ G{m) be the natural embedding. Let / e 
I{G{m)) be a test function and f\G{n) be its restriction to G{n) , which is clearly 
an element of I{G{n)) . Let us write the expansion (3.1) for / and f\G{n): 

(/|G(n))'(F)= ^ (f^|„,/|G(n))T^|,(F), F G s(n) . 

Assume X — Y ^ g{n) . Then the left-hand sides of both expansions coin- 
cide. On the other hand, by the definition of the polynomial functions and the 
stability property of the Schur functions, we have 

if n. 

It follows 

(f^l„,/|G(n)) = (f^l^,/), /e/(G(m)), l{fi)^n. 
More generally, for any / e C [G] we have 

(T^l^, /) = (f ^1^, /#) = (f ^1^, f* I G{n)), 
which is equivalent to the claim of the theorem. □ 

Remark 3.2. It was shown in [001], that the coherence property of the basis 

is equivalent to the following relation satisfied by the polynomials s* : for any 

partition jj, of length ^ n and any signature A of length n + 1 , 



^A___ .;|^^,(Ai, . . . , A.+i) = <|n(Ai, . . . , An), (3.2) 



where A denote a signature of length n and A ^ A means the Gelfand-Tsetlin 
interlacing condition 

Ai ^ Ai ^ A2 ^ • • • ^ A„ ^ An ^ 

A similar relation, which is equivalent to the coherence property of {T^} , holds for 
the polynomials t* : 

^Jf^ tX^Mu A,.0 = ElXA : X.1 ^ «:,„(A„ . . . , A„) . (3.3) 

Here A is a partition of length ^ n -|- 1 , ^ is a partition of length ^ n , x\ is 

one of the characters of G{n + 1) (i.e., X^^*'^"'''"^'* , Xa'^'^^^^^ °^ Xa^"^^"*), Xa has 
the same meaning (but for the group G{n) of rank n), and [xa : Xa] denotes the 
multiplicity of xa in the decomposition of xa as restricted to G{n) C G{n + 1) ^. 

In [001, §10] we gave a direct derivation of the relation (3.2). The relation (3.3) 
can be directly verified by a similar (but more complicated) argument. This gives 
another approach to the coherence property. 



^This multiplicity is given by the well-known 'branching rules' for symplectic and orthogonal 
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§4. A DISTINGUISHED BASIS IN 

In this section, we study a distinguished basis in C S{q) (see section for 
the definition of I{g) ). 

We equip g with an invariant inner product: for any X,Y & q, 



trXY, for the series A, 

i tr XY, for the series C, B, D . 



We shall identity g with its dual space q* by making use of the product ( • , • ) . 
This will allow us to interpret S{q) as the algebra of polynomial functions on q 
and then elements of I{g) will become invariant polynomial functions on g . 

Assume X ranges over g. In the case of the series A we shall denote by 
xi,...,Xn the eigenvalues of X. In the case of the series C, D the eigenvalues 
of X may be written as ixi, . . . , ±Xn , and for the series B one has to add one . 
In this notation we set 



Si^{X) = Sf^iniX) = S^{xi, ...,Xn), 

Tf,{X) =T^\niX) =t,^(xi,...,Xn) = Sf,{xl,...,xl). 

Clearly, {S^} or {T^} is a homogeneous basis in I{g) . 

Note that I{g) may be identified with grZ{g), the graded algebra associated 
with the fibered algebra Z{g) . The next claim is immediate from definitions. 

Proposition 4.1. Under the identification I{g) — grZ{g), each basis element 
Sjj, G I{g) or e I{g) coincides with the leading term of the basis element e 
Z{g) or e ^io) ; respectively. □ 

Now write g{n) instead of g and consider the chain • • • ^ g{n) ^ g{n+l) ^ ■ ■ ■ 
of natural embeddings that we already discussed in §3. Note that these embedding 
are isometric with respect to the inner product ( • , • ) . Therefore, for each couple 
n < m there is a natural projection g{m) and so an algebra morphism 

S{g{m)) — > S{g{n)) , which in turn induces an algebra morphism 

ProJnm : ^(0(^)), n<m. 

Proposition 4.2 (Stability of the bases {S^} , {T^^}). Assume n <m. If l{ii) ^ n 
then 

If n < l{fi) ^ m then the result is zero. 

Proof. This follows at once from the stability property of the Schur polynomials. □ 
On the other hand, for any n < m we can define linear maps 



Avr^m: I{&{n)) I{d{m)) 
(the averaging operators) just in the same way as for the invariants in the enveloping 
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Proposition 4.3 (Coherence property). Assume n <m and l{iJ,) ^ n. Then 

AVr„ml-5^|n; - ^^^^^^ -S^lm, AVr„^^J^|„; - ^^^^^^^ J/x|m- 

/n particular, the averaging operators are injective. 

Proof. This foUows from Theorem 3.1 and Proposition 4.1. 

Theorem 4.4 (Characterization of the basis). Let n be fixed and fi range over 
partitions of length ^ n. The basis elements S^^^ {for the series A) or T^\n {for 
the series C, B, D) are the unique, within a scalar multiple, elements of I{g{n)) 
that are eigenvectors of all linear mappings 

Proj^^ o AvTnm ■ I{d{n)) I{din)), m = n + l,n + 2, . . . . 



Proj^^(Avr^^(^^l„)) = -^-1-^ 5^,^, 
Projn..(Avr.^(T^I„)) = T^,, , 



Proof. By Propositions 4.2 and 4.3, we have for any m > n 

c{n,ii) 
c{m, n) 
c±{n,n) 
c±{m,ij) 

Thus, each basis element is an eigenvector for all the maps Proj^^ o Avr„^ . It 
remains to show that the eigenvalues corresponding to two different basis vectors 
are distinct for certain m. But this follows from the explicit expressions for c(n, /x) 
and c±(n, //) given in §1. □ 

In [001, (2.10)], we obtained an explicit expression of the basis elements S^\n £ 
I{Q\{n)) through the matrix limits Eij e st(n), 1 ^ z,j ^ n: 

Proposition 4.5. Let ^ he a partition of length ^ n, k = , and be the 

irreducible character of the symmetric group &{k) that is indexed by fi . Then 

n 

S^\n = {kl)-' J2 E x'^(^)^nM,)---£^i.v.). 

il,...,ife = l se&(k) 

We shall give now a similar formula for the series C, B, D. Assume g = g{n) is of 
type C, B, D and realize it as an involutive subalgebra in qI{N, C) , where N — 2n 
or A?" = 2n + 1 , as indicated in §1. We shall assume the canonical basis in is 
labeled by the numbers i = —n , —n +l,...,n — l,n, where is included for the 
series B only, and we put 

where 9ij = 1 for the series B, D and 9ij = sgn(i) sgn(j) for the series C. Note 
that for our embedding g{n) ^ dl{N, C) the elements F^- form a basis in g{n) . 

Given a partition k we define a central function (f^ on the symmetric group 
&{2k) as follows. If a permutation s e &{2k) is such that all its cycles are even 
(so that the circle type of s may be written as 2p for a certain p\- k) then we put 

where Xs denotes the value of the irreducible character permutation 
with cycle type p. Otherwise (i.e., if s e &{2k) has at least one cycle of odd 

length) v?'^(s) = . 
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Proposition 4.6. Assume q = Q{n) is of type C,B,D . Let fi be an arbitrary 
partition of length ^ n and k = \fj,\ . Then 



n 



^ '' ii,...,i2k = -n sGS(2fe) 

Proof. Let us view T^|„ as an invariant polynomial function T^|„(X), where X 
ranges over g(n) and denote by ±xi, . . . , ±a;n the eigenvalues of X (for the series 
B we exclude the zero eigenvalue). We shall use standard notation of Macdonald's 
book [M2]: pp are the power sums and 

Zp = I'^'hl 2^^k2\ ■■■ for p = {1^^2^^ ■■■). 

By definition of the basis {T^} we have 

phk 

= Yl ^p^'^~^''^^XpP2p{xi, -Xi,..., Xn, -Xn) 
p\-k 

— ^ ] ^2p XpP2p{xij —Xi, . . . , Xn, —Xn) ■ 
p\-k 

Let s range over elements of ©(2/c) with even cycle type 2p (where p \- k 
depends on s ) . Then the latter expression can be written as 

T,j,\n{X) = ^p^'{s)p2p{xi, -Xi, . . . , Xn, -Xn) ■ 

On the other hand, for any (even) m , the invariant polynomial function 

Pm{X) = tr(X"') = Pni{xi, -Xi, ...,Xn, -Xn) 

corresponds to the element 

n 

Yl Fi.i^Fi^i, ■ ■ ■ Fi^i, e /(0(n)), 
which also can be written as 

n 

V F F- 

for any cyclic permutation s of the indices 1, . . . , m. This concludes the proof. □ 
There exists another characterization of the bases {5"^}, {Tju}, which is based 
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Proposition 4.7. Define a classical group G containing G as follows. 

If G = GL{n,C) then G = GL{n,C) x GL{n,C) and the embedding G ^ G 
has the form g ^ {g, {g')~^) ■ 

If G is one of the groups SO{2n+ 1,C) or Sp{2n,C), SO{2n,C) then G = 
GL[N, C) , where N = 2n + 1 or N = 2n , respectively. 

Then the adjoint action of G in q can he extended to a linear action of the group 
G D G such that the induced representation of G in the symmetric algebra S{q) 
is a multiplicity free polynomial representation. 

Proof. Assume G = GL(n, C). Using the bijection Eij ^ Cj ® Cj, where {cj} 
stands for the natural basis of , we identify £| = £|[(n, C) with The 
action of the group G — GL{n, C) x GL{n, C) in the vector space g = C^^C"^ is the 
natural one. Clearly, its restriction to the subgroup G = {{g, {g')~^)} is equivalent 
to the adjoint action. It is well-known that action of the group GL{n, C) x GL{n, C) 
in S'(C"' (8) C") is a multiplicity free polynomial representation, see, e.g., Zelobenko 
[Z,§56] or Howe [H, 2.1]. 

Assume G = Sp{2n, C) and put N = 2n. One can identify Q = sp{2n, C) with 
S'^{C^) in such a way that the adjoint action of G in g will coincide with the 
natural action of G = GL{N, C) in S^{C^) , restricted to G C G . It is well known 
[H, 3.1] that the natural action of GL{NX) in S{S'^{C^)) is a multiplicity free 
polynomial representation. 

Assume now G = SO{N,C) , where N — 2n + 1 or N — 2n. Then one can 
identity g with A^(C^) in such a way that the adjoint action of G in g will 
coincide with the restriction to G of the natural action of the group G = GL{N, C) 
in A2(C^). On the other hand, it is weU known [H, 3.8] that ^(A^C^) is a 
multiplicity free polynomial GL ( A?", C) -module. 

Finally, note that in this argument, we could replace the special orthogonal group 
SO{N, C) by the complete orthogonal group 0{N, C) . □ 

The next result provides us with an alternative characterization of the bases 
{S,}, {T,}. 

Theorem 4.8. Let G be a classical group and G D G be as in Proposition 4.7. 
The basis elements G /(g) or T^ G /(g) are the unique {within a scalar mul- 
tiple) elements of /(g) C S{q) that generate, under the action of G , irreducible 
submodules of S{q) . 

Proof. Examine the case G = GL{n, C) . The representation of the group G = 
GL{n, C) X GL{n, C) in the space S{C"' ® C"') is decomposed in the direct sum of 
the irreducibles Vp|„ ® Vp|„ , where /x ranges over partitions of length ^ n (see [Z, 
§56] or [H, 2.1.2]). Each component V^i^ V^in clearly contains a unique (within 
a scalar multiple) vector 5'^, invariant under the subgroup G = {{g, {g')~^)} ■ It 
is also evident that the elements S'^ can be characterized as unique elements of 
/(g) generating irreducible G-submodules of S{g) . Thus, we have to check that 
S'l^ — Sfj, (within a scalar multiple). 

By Theorem 4.4, it suffices to show that each S'^ is an eigenvector for the maps 
Proj„^ o Avr„^ , m = n + l,n + 2, . . . , but this follows from the fact that 



Avr^^(V^I^) C V^i^, Proj^^(V^i^) = V^|„, 
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For other groups G the argument is similar. 

Let G = Sp{2n,C) and put N = 2n. The representation of G = GL{N,C) in 
S{S^{C^)) is the direct sum of the irreducibles Vm\n i where M is a Young diagram 
with even rows (the number of rows does not exceed A^); on the other hand, an 
irreducible polynomial G'L(A^, C) -module Vm\n contains a vector invariant under 
Sp{2n, C) , where n = N/2, if and only if all the columns of M are even, and then 
such a vector is unique within a scalar multiple (see [H, 3.1] or [M2, VII, (6.11)]). 

Thus, an irreducible component Vm\n C S{S'^{C'^"')) has a nonzero (and then 
one-dimensional) intersection with I{q) if and only if M can be written as 

M = 2/i U 2/i = (2//1, 2//1, 2/X2, 2//2, • • • , S/Xn, 2//„), 

where is a partition of length ^ n. Let be any nonzero element of the 
one-dimensional space V'2^u2^i|2n ^ (fl) • The same argument as above shows that 
= , within a scalar multiple. 

(Note, however, a difference with the case G — GL{n, C) examined above. For 
G = GL{n, C) , we saw that each irreducible G-submodule contained a G-invariant, 
while now only a part of components possess G -invariants.) 

Now let G = SO{N, C) , where N = 2n+l or N = 2n , and remark that in both 
cases I{g) coincides with the subspace of S{g) formed by the G' -invariants, where 
G' = 0(N,C). The representation of G = GL{N,C) in the space S{A'^{C^)) 
decomposes into the direct sum of irreducible modules Vm\n 7 where M is a diagram 
with even columns. On the other hand, a G' -invariant in Vm\n exists (and then 
is unique, within a scalar multiple) if and only if M has even rows (see [H, 3.8] 
or [M2, VII, (3.14)]). Thus we again obtain that M has the form 2/x U 2/x and 
conclude the proof as above. □ 

§5. A RELATIONSHIP BETWEEN THE BASES IN Z{q) AND /(g) 

Here we exhibit a linear isomorphism S{g) U{q) , called the special sym- 
metrization, which maps I{g) onto Z{q) and takes the canonical basis in /(g) 
to that of Z{q) . The main result is Theorem 5.2. We reduce it to Proposition 
5.3 which in turn is reduced to Proposition 5.4. These two propositions are of 
independent interest 

We shall heed the concept of generalized symmetrization proposed by Olshanski 
[02]. Let us identify the algebra U (g) with a subspace of the dual of Oe{G) (see the 
beginning of §2). Similarly, we shall identify S{q) with a subspace of the dual space 
to Oq (q) , the space of germs of holomorphic functions defined at a neighborhood 
of the origin G . 

Assume we are given a map F: q ^ G with the following properties: 

(i) F is holomorphic and defined in a neighborhood of the origin, invariant 

under the adjoint representation; 
(n) F takes e 5 to e e G ; 

(iii) the differential of F at the origin is the identical map Q ^ q; 

(iv) F is equivariant with respect to the group G acting by the adjoint repre- 
sentation in g and by conjugations on itself. 

Then F induces an isomorphism of local rings Oo{q) — > Oe{G) , which by duality 
determines a linear isomorphism 
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Note that this construction has a sense for any complex Lie group; in fact it also 
works for formal groups. 

When F is the exponential map, the corresponding map a coincides with the 
standard symmetrization map. Following [02], in the general case we call a a 
generalized symmetrization. 

A generalized symmetrization a shares a number of properties of the standard 
symmetrization. In particular, a preserves leading terms and also induces a bijec- 
tion between invariants of the group. 

Thus, for a classical group G, a induces a linear isomorphism /(g) Z[q) (for 
the series D we shall assume F is invariant under G' D G). 

Theorem 5.1 ([001, Theorem 14.1]). Assume q = 0[(n, C) and F{X) = 1+X , 
and let a he the corresponding generalized symmetrization. Then 

(^{Sij,\„) = Spin for any £(/x) ^ n . □ 

This generalized symmetrization was introduced in [01]; then it was used in 
the note [KO] and called the special symmetrization. We aim to find a similar 
generalized symmetrization for the series C, B, D. 

Theorem 5.2. Let g = Q{n) he any classical Lie algehra of type C,B,D , realized 
as an involutive subalgehra of gl{N, C) , where N = 2n or N = 2n + 1 . Define 
F: Q ^ G by the relation 

F{Xy/^ - F{X)-^/^ ^ X for X eg, 

i.e., 

F{X) = 1 + X^/2 + ((1 + Xy2f - 1)^/2 = 1 + xy2 + X{1 + X^A^/^ 
= 1+X + --- , 

where the eigenvalues of the matrix X are supposed to be sufficiently small. 
Let a be the corresponding generalized symmetrization. Then 

(^iT^\n) = ^fx\n for any /i, l{ii) ^ n. 

Note that F is indeed a well-defined (local) map from q to G . We call a the 

special symmetrization for the classical Lie algebras of type C, B, D. We refer to 
[02] for explicit formulas for a: S{q) U{q) and its inverse : U{q) S{g) . 

Note that the proof of the theorem given below differs from the argument used 
in [001] for the series A. On the other hand, the argument given below also holds 
for the series A. We do not know if the approach of [001, Theorem 14.1], can be 
transferred to the series C, B, D. 

Proof. Let (p e Oe{G) stand for a test element. By definition, we have to prove 
the equality 

where the brackets on the left-hand side denote the pairing between U{q) and 
Oe{G) while the brackets on the right-hand side denote the pairing between S{q) 
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Let 

accr" c OeCc) and Oo(0r"cco(0) 

denote the subspaces of invariants with respect to the action of the group G (or 
the group G' , for the series D). Choose a compact form K C G (or K' C G'). 
Averaging over K (or K' ) determines an invariant projection 

Oe{G) ^ C»e(G')'"". 

Using it we reduce our problem to the case (p e Oe{Gy^^ . 

We shaU apply the binomial formula (Theorem 1.2) which can be conveniently- 
written as 

where (p e OeiGy^^ . Indeed, if (p has the form (p = Xx/xx{^)i where XA is one 
of the characters Xa*^^"^ » Xa"^^""*"^^ j X°\'^^^ then the above relation just coincides 
with the binomial formula, because of the definition of the map F and the relation 

(T^|n, X\/Xx{e)) = |„(Ai, . . . , An) . 

Further, since the linear span of the characters xa is dense in Oe(G)'"^ with 
respect to the adic topology defined by the unique maximal ideal of Oe{Gy^^ , our 
expansion holds for any (p e Oe{Gy"^^ . 

On the other hand, any element ip e Oo(s)'"^ can be uniquely written as a series 

converging in the adic topology of Oo(0)'"^ ■ By taking ip = (p o F and comparing 
the both expansion we reduce the problem to the following relation 

c±{n,ii} 

Finally, without loss of generality we may assume tp = T^\n for a certain partition 
u of length ^ n, because {T^i^} is a topological basis in Oo(b)^"^- 
Thus, the final reduction of the problem is the following claim. 

Proposition 5.3. Let fj, , v be partitions of length ^n. Then 



Comment. The brackets in the left-hand side can be understood in two different but 
equivalent ways. First, they represent the pairing between a distribution supported 
at the origin and a polynomial function, i.e.. 
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where 9(T^|„) stands for the differential operator on q with constant coefficients 
that corresponds to r^|„ e . Second, the brackets may denote the canonical 
extension to S (q) of the inner product on q . 

Proof. Probably, the proposition could be proved starting from the explicit expres- 
sion for T^i^ given in Proposition 4.6 but we prefer to use another argument. 

Let us extend to S{q) the inner product (•, •) in g. Then one can associate 
with {S{q), {• , • )) a reproducing kernel S{X, Y) , where X,Y E Q. By definition, 

£{x,Y) = j2Mx)raiy), 

a 

where {i^a} is an arbitrary homogeneous basis in S{g) and {i^^} is the dual basis. 
Note that S{X,Y) does not depend on the choice of the basis. 

Similarly, to the inner product space (/(g), ( • , • )) also corresponds a reproduc- 
ing kernel J-'{X, Y) . 

Between these two kernels there is an evident relation 

T{X,Y) = J £{X,Adu-Y)du = J £{Adu- X,Y)du, 

where the integral is taken over the compact form K C G (or K' C G') equipped 
with the normalized Haar measure. 

Further, taking as {V'a} the basis of monomials formed from a basis in g, we 
obtain that 

whence 

On the other hand, the claim of the proposition means that 

because {T^\n} is a homogeneous basis in /(g) . 

Thus, we have reduced our problem to the following claim. 

Proposition 5.4. Let u range over a compact form K <Z G {or K' d G' , for the 

series D) and let du denote the normalized Haar measure. The following expansion 
holds 



/ (x,Ad..y)^^^ y T^\n{X)T^\n{Y) ^ X,reg = g(n). (5.1) 



Proof. We shall show that this can be obtained from the binomial formula (Theorem 
1.2) by a limit transition. 

First of all, without loss of generality we may assume X and Y are diagonal 
matrices with diagonal entries ±Xi and ±.yi , respectively (as usual, we omit the 



24 



ANDREI OKOUNKOV AND GRIGORI OLSHANSKI 



T^\n{X) by t^iriixi, ■■■,Xn) and T^|n(^) by t/,|n(2/i, ■ ■ ■ , l/n) ■ After this the right- 
hand side becomes very similar to the right-hand side of the binomial formula: the 
only difference is that in the binomial formula, we have i^|^(Ai, . . . , A^) instead of 

Let us compare now the left-hand sides of both formulas (in the discussion below 
one has to replace K by K' for the series D). 

In the binomial formula (1.8) occurs the normalized irreducible character X\/xxi^) j 
which can be viewed as a spherical function for the Gelfand pair [K x K, K) . The 
left-hand side of formula (5.1) also can be viewed as a spherical function for a 
Gelfand pair. Namely, this pair consists of the Cartan motion group K tK i (the 
semidirect product of K and its Lie algebra t viewed as a X -module) and its 
subgroup K . The vector x = (ixj) can be considered as the argument of the 
spherical function and y = (±2/i) is the parameter. 

Now we shall use the well-known relation between the spherical functions of a 
symmetric space (say, of compact type) and the spherical functions of the corre- 
sponding Cartan motion group (see, e.g., Dooley and Rice [DR]). 

In our context, this relation looks as follows. Let £ be a small parameter which 
then will tend to . Assume g E K has the form g = 1 +eX + 0{e) , where X E 
and the partition A has the form A = e~^y + 0{e~^) Then in the limit £ — > the 
normalized character indexed by A turns into the spherical function for the Cartan 
motion group, indexed by y . 

Since the top homogeneous component of the polynomial coincides with 
, the right-hand side of the binomial formula (1.8) will turn, after this limit 
transition, into the right-hand side of formula (5.1). 

This concludes the proof of Proposition 5.4 and at the same time of Proposition 
5.3 and of Theorem 5.2. 

§6. Appendix: Bispherical functions on 5'p(2n, C) \ GL(2n, C)/0(2n, C) 

In this section we discuss a curious fact suggested by the results of §4. 
Consider two Gelfand pairs (G, G) : 

{GL{N, C), 0{N, C)), N = 2n + l or N = 2n, 

and 

{GL{N,C),Sp{N,C)), N = 2n. 

The spherical functions of these pairs are matrix elements g h- > {V{g)^,^), where 
g ranges over G , V as an irreducible finite-dimensional G-module, and ^ ia a G- 
invariant vectors. As is well-known these spherical functions can be identified (with 
an appropriate parametrization of the double G-cosets in G) as Jack polynomials 
pji"'^ in n variables, where the parameter a takes the value a = 2 for the former 
pair and the value a = 1/2 for the latter one, and fx ranges over partitions of 
length ^ n . 

^ Since the both sides of (5.1) are invariant with respect to the action of the hyperoctahedral 
group on yi , . . . , y„ , we may assume J/i ^ • • ■ ^ J/n ^ , so that the approximation of the vectors 
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The aim of this Appendix is to examine what happens when these two pairs are 
"mixed" in the following sense. We set 

G = GL(2n,C), Gi = 0(2n,C), G2 = Sp{2n,C), 

and assume V is an irreducible G -module admitting both a Gi -invariant ^ and a 
G2 -invariant rj. Then the dual module V* also possesses a G2 -invariant 77* and 
we form the matrix element 

which we call a bispherical function. We shall calculate (f in terms of a natural 
parametrization of the (G2, Gi) -cosets in G and we shall see that (/? is a Schur 
polynomial (note that the Schur polynomials are the Jack polynomials with a = 1). 

Without loss generality we may assume F is a polynomial module. We shall 
specify the embeddings Gi ^ G as indicated in §1. 

Theorem 6.1. Let V be an irreducible polynomial representation of G = GL{2n, C) 
admitting both a Gi -invariant ^ and a G2 -invariant rj (where Gi = 0(2n, C) and 
G2 = Sp{2n,C)), i.e., 

where fj. is a partition of length ^ n and 

2/x U 2/x = (2/xi, 2/xi, . . . , 2/Xn, 2/Xn) . 
Then the spherical function 

^ti{g) = (^2pU2p|2n(fi')^,^7*), 

where g & G and 77* stands for a G2 -invariant in V* , is proportional to the Schur 
polynomial in n variables under a suitable parametrization of the double cosets 
G2gGi in G . 

First proof. Thus proof is based on Theorem 4.8. Let g ^ g' denote transposition 
of 2n X 2n matrices that corresponds to the symmetric form M preserved by the 
subgroup Gi = 0{2n, C) , see §1 for the choice of M . I.e., g ^ g' is transposition 
with respect to the secondary diagonal. The map g ^ gg' defines a bijection 
between the cosets gGi C G and (nondegenerate) 2n x 2n matrices, symmetric 
relative the secondary diagonal so one can write 

where i/j^ is a polynomial function on symmetric matrices 

The group G acts on symmetric matrices, and under this action ifj^ is specified 
by two properties: first, it transforms according to V2fj,[j2n and, second, it is G2- 
invariant. 

Next, we identity the space of symmetric matrices with the Lie algebra sp{2n, C) 
or, which is better, with its dual space. Under this identification, the functions V'/u 
become elements of the space I{sp{2n, C)) , and application of Theorem 4.8 implies 
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Given a n -tuple {xi, . . . , x^) of nonzero complex numbers, form a diagonal 
matrix, 

g{x) = (iiag{xi, ... ,Xn,Xn, ... ,xi) . 

Then 

9ix)i9ix)y = diag (xj, . . . , xl, xl, . . . , xl) . 

Under the above identification between symmetric matrices and elements of the 
(dual space to) the Lie algebra sp{2n, C) the latter matrix turns into 

A- { 2 2 2 2\ 

aiag [Xl, . . . , x^, —x^, . . . , —Xl) 

which is an element of the Cartan subalgebra of sp{2n, C) . 
It follows 

"pMx)) = '4)^,{g{x){g{x))') = const s ^{x\, x^) , 

by the definition of the elements for the series C. To conclude the proof we 
remark that each double {G2, Gi) -coset in G has the form G2g{x)Gi with a certain 

X = {xi, ...,Xn). □ 

Second proof {sketch). One can choose a compact form G^ C G (isomorphic to 
U (2n) ) such that = fl Gj is a compact form of G^ for z = 1, 2 . We also can 
arrange so that the matrices g{x) be in G" provided |a;i| = • • • = j^nl = 1 • Then 
each (G2, Gi)-coset in G" is of the form G2g{x)Gi , so that we can use xi, . . .,Xn 
as parameters of the (G2, G^)-cosets in G'^ . 

A direct computation shows that the "radial part" of the Haar measure of the 
compact group G" , expressed in these parameters, has the density 

w{x) = const Y\ ~ ^jP- 

It follows that the bispherical functions are symmetric orthogonal polynomials in 
xf, . . . ,xf^ with weight w{x) . 

Finally, analysis of the weights of V2^u2^|2n shows that the leading term of 
the bispherical function (with respect to the lexicographic order on monomials in 
xf, . . . ,xfi^) is equal to 

const Xi'^'^ ■ ■ ■ x^''" . 

Hence our orthogonal polynomials coincide with the Schur polynomials s^(a;f , . . . , ). □ 
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